United States Patent and Trademark Ofhce 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark OtBce 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



10/586,319 



07/14/2006 



26339 7590 10/09/2009 

MUIRHEAD and SATURNELLI, LLC 
200 FRIBERG PARKWAY, SUITE 1001 
WESTBOROUGH, MA 01581 



HICKS, CHARLES V 



PAPER NUMBER 



DELIVERY MODE 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



KJtSiVrXS nvrliyjts OUff Iff fcff Jr 


Application No. 

10/586,319 


Applicant(s) 

INOUE, MIKIO 


Examiner 
CHARLES HICKS 


Art Unit 

2629 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
eamed patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^ Responsive to communication(s) filed on 31 July 2009 . 
2a )□ This action is FINAL. 2b)|3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 4.6.7.12.13 and 15-18 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) IEI Claim(s) 4.6.7.12.13 and 15-18 is/are rejected. 
/)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 14 July 2006 is/are: a)^ accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)S Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)^ All b)n Some * c)^ None of: 

1 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftspereon's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26'(Rev^'o8-0^^ 



Office Action Summary 



Part of Paper No./Mail Date 20091001 



Application/Control Number: 10/586,319 
Art Unit: 2629 



Page 2 



DETAILED ACTION 

The communication is responsive to amendments filed 07/31/2009. Claims 1-3, 
5, 8-1 1 and 14 have been cancelled. Claims 15-18 are new. Currently claims 4, 6-7, 
12-13, and 15-18 are pending. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 4, 6, 7, 15 and 18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Rafii et al. (6,614,422) in view of Kloba (US 7,263,547). 

In reference to claim 4, Rafii teaches a mobile communication terminal (Rafii, Fig. 
1A, 80), comprising: 

image projection means for projecting one of a plurality of predefined operation- 
plane images that displays virtually an operation-plane of an operation device operated 
by users (Rafii, Fig. 1A, 145; col. 4, II. 27-33); 

operation detection means for detecting operation on the operation-plane image 
projected by the image projection means (Rafii, Fig. 1A, 20; col. 10, II. 27-34); 
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data processing means for performing a predetermined data process based on 
the detection result of operation detected by tlie operation detection means (Rafii, Fig. 
3; col. 7, II. 16-18); 

wherein the image projection means projects an operation-plane image 
corresponding to recognition function designated by designation information received 
from an application execution management means, from among the plurality of 
predefined operation-plane images (Rafii, col. 4, II. 27-33; projection of a grid or image); 

and the operation detection means has a plurality of kinds of mutually different 
recognition functions to recognize operation content by at least one of position, direction 
and movement of an operation object on the plurality of predefined operation-plane 
images (Rafii, col. 12, II. 33-47), 

and detects operation on the operation-plane image by using the recognition 
function designated by designation Information received from the application execution 
management means (Rafii, col. 10, II. 27-34). 

Rafii however fails to teach application execution management means for 
managing application program execution environment of an application program 
selected from a plurality of application programs that is downloaded via a mobile 
communication network. 

Kloba discloses a system for customizing content on a mobile device, analogous 
in art with that of Rafii, containing application execution management means for 
managing application program execution environment of an application program 
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selected from a plurality of application programs that is downloaded via a mobile 
communication network (Kloba, col. 4, II. 37-41; col. 7, II. 5-9; col. 11, II. 15-21). 

At the time the invention was made, it would have been obvious to one having 
ordinary skill in the art to modify the mobile communication terminal of Rafii to include 
application execution management means for managing application program execution 
environment of an application program selected from a plurality of application programs 
that is downloaded via a mobile communication network, as taught by Kloba. 

As one of ordinary skill in the art would appreciate, the suggestion/motivation for 
doing so would have been to enable the user to run multiple applications on a mobile 
device while on-line or off-line (Kloba, col. 1, II. 36-39). 

In reference to claim 6, Rafii teaches a mobile communication terminal (Rafii, Fig. 

1A, 80), comprising: 

image projection means for projecting one of a plurality of predefined operation- 
plane images that displays virtually an operation-plane of an operation device operated 
by users (Rafii, Fig. 1A, 145; col. 4, II. 27-33); 

operation detection means for detecting operation on the operation-plane image 
projected by the image projection means (Rafii, Fig. 1A, 20; col. 10, II. 27-34); 

data processing means for performing a predetermined data process based on 
the detection result of operation detected by the operation detection means (Fig. 3; col. 
7, II. 16-18); 
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wherein tlie image projection means projects an operation-plane image 
corresponding to recognition function designated by designation information received 
from an application execution management means, from among the plurality of 
predefined operation-plane images (Rafii, col. 4, II. 27-33; projection of a grid or image); 

and the operation detection means has a plurality of l<inds of mutually different 
recognition functions to recognize operation content by at least one of position, direction 
and movement of an operation object on the plurality of predefined operation-plane 
images (Rafii, col. 12, II. 33-47), 

and detects operation on the operation-plane image by using the recognition 
function corresponding to the operation-plane image designated by designation 
information received from the application execution management means (Rafii, col. 10, 
I. 27-34). 

Rafii however fails to teach application execution management means for 
managing execution environment of an application program selected from a plurality of 
application programs that is downloaded via a mobile communication network. 

Kloba discloses a system for customizing content on a mobile device, analogous 
in art with that of Rafii, containing application execution management means for 
managing execution environment of an application program selected from a plurality of 
application programs that is downloaded via a mobile communication network (Kloba, 
col. 4, II. 37-41; col. 7, II. 5-9; col. 11, II. 15-21). 

At the time the invention was made, it would have been obvious to one having 
ordinary skill in the art to modify the mobile communication terminal of Rafii to include 
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application execution management means for managing execution environment of an 
application program selected from a plurality of application programs that is downloaded 
via a mobile communication network, as taught by Kloba. 

As one of ordinary skill in the art would appreciate, the suggestion/motivation for 
doing so would have been to enable the user to run multiple applications on a mobile 
device while on-line or off-line (Kloba, col. 1, II. 36-39). 

In reference to claim 7, Rafii teaches a mobile communication terminal (Rafii Fig. 
1A, 80), 

comprising: image projection means for projecting one of a plurality of predefined 
operation-plane image that displays virtually an operation-plane of an operation device 
operated by users (Rafii, Fig. 1A, 145; col. 4, II. 27-33); 

operation detection means for detecting operation on the operation-plane image 
projected by the image projection means (Rafii, Fig. 1A, 20; col. 10, II. 27-34); 

data processing means for performing a predetermined data process based on 
the detection result of operation detected by the operation detection means (Fig. 3; col. 
7, II. 16-18); 

memory means for storing a plurality of image data corresponding to each one of 
the plurality of predefined operation-plane images (Rafii, col. 12, II. 48-53); 

and instruction generation means for generating an operation-plane image 
selection instruction in accordance with content of the selected application program 
(Fig. 3; col. 7, II. 16-18); 
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wherein the image projection means selects an image data from the plurality of 
image data memorized in the memory based on the operation-plane image selection 
instruction generated by the instruction generation means, and projects the operation- 
plane image of the selected image data (Rafii, col. 4, II. 27-33; projection of a grid or 

image); 

and an application execution management means performs a data process 
corresponding to operation detected by the operation detection means in accordance 
with the content of the application program during execution of the selected application 
program (Rafii, col. 10, II. 27-34) . 

Rafii however fails to teach application execution management means for 
executing an application program selected from a plurality of kinds of application 
programs that is downloaded via a mobile communication network. 

Kloba discloses a system for customizing content on a mobile device, analogous 
in art with that of Rafii, containing application execution management means for 
executing an application program selected from a plurality of kinds of application 
programs that is downloaded via a mobile communication network (Kloba, col. 4, II. 37- 
41; col. 7, II. 5-9; col. 11, II. 15-21). 

At the time the invention was made, it would have been obvious to one having 
ordinary skill in the art to modify the mobile communication terminal of Rafii to include 
application execution management means for executing an application program 
selected from a plurality of kinds of application programs that is downloaded via a 
mobile communication network, as taught by Kloba. 
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As one of ordinary skill in the art would appreciate, the suggestion/motivation for 
doing so would have been to enable the user to run multiple applications on a mobile 
device while on-line or off-line (Kloba, col. 1 , II. 36-39). 

In reference to claim 15, Rafii teaches a mobile communication terminal (Rafii, 
Fig. 1A, 80), 

comprising: an image projector that projects one of a plurality of predefined 
operation-plane images that displays virtually an operation-plane of an operation device 
(Rafii, Fig. 1A, 145; col. 4, II. 27-33); 

an operation detector that detects operation on the operation-plane image 
projected by the image projector (Rafii, Fig. 1A, 20; col. 10, II. 27-34); 

a data processor that performs a predetermined data process based on the 
detection result of operation detected by the operation detector (Rafii, Fig. 3; col. 7, II. 
16-18); 

wherein the image projector projects an operation-plane image corresponding to 
recognition function designated by designation information received from an application 
execution management device, from among the plurality of predefined operation-plane 
images (Rafii, col. 4, II. 27-33; projection of a grid or image); 

and the operation detector has a plurality of kinds of mutually different 
recognition functions to recognize operation content by at least one of position, direction 
and movement of an operation object on the plurality of predefined operation-plane 
images (Rafii, col. 12, II. 33-47), 
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and detects operation on the operation-plane image by using the recognition 
function designated by designation information received from the application execution 
management device (Rafii, col. 10, II. 27-34). 

Rafii however fails to teach an application execution management device that 
manages an execution environment of an application program selected from a plurality 
of application programs that is downloaded via a mobile communication network. 

Kloba discloses a system for customizing content on a mobile device, analogous 
in art with that of Rafii, containing application execution management device that 
manages an execution environment of an application program selected from a plurality 
of application programs that is downloaded via a mobile communication network (Kloba, 
col. 4, II. 37-41; col. 7, II. 5-9; col. 11, II. 15-21). 

At the time the invention was made, it would have been obvious to one having 
ordinary skill in the art to modify the mobile communication terminal of Rafii to include 
an application execution management device that manages an execution environment 
of an application program selected from a plurality of application programs that is 
downloaded via a mobile communication network, as taught by Kloba. 

As one of ordinary skill in the art would appreciate, the suggestion/motivation for 
doing so would have been to enable the user to run multiple applications on a mobile 
device while on-line or off-line (Kloba, col. 1 , II. 36-39). 
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Claim 18 is rejected as being dependent on rejected claim 15 as discussed 
above and further, Rafii teaches further comprising: a memory that stores the plurality of 
predefined operating-plane images (Rafii, col. 12, II. 48-53). 

Claims 12, 13, 16 and 17 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Rafii et al. (6,614,422) as modified by Kloba (US 7,263, 547) and 
further in view of Lieberman (US 2002/0075240). 

Claim 12 is rejected as being dependent on claims 4, 6, or 7 as discussed above 
and further, Rafii modified by Kloba fails to teach a mobile communication terminal 
wherein the mobile communication terminal is configured by using a light source, a 
spatial light modulation unit for modulating light output from the light source, and an 
optical system for projection imaging that projects by imaging a light image output from 
the spatial light modulation unit on an external projection screen, the mobile 
communication terminal comprises an optical system for diffused illumination for 
homogenously illuminating by diffusing light output from the light source to an external 
illumination plane, and the light source and the spatial light modulation unit are both 
shared to generate a light image subject to projection and generate a light subject to 
diffused illumination. 

Lieberman discloses a virtual data entry device, analogous in art with that of Rafii 
modified by Kloba, wherein the mobile communication terminal is configured by using a 
light source, a spatial light modulation unit for modulating light output from the light 
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source, and an optical system for projection imaging that projects by imaging a liglit 
image output from the spatial light modulation unit (Lieberman, Fig. 28; pg. 10, par. 184) 

on an external projection screen (Lieberman, Fig. 29; pg. 10, par. 186), 

the mobile communication terminal comprises an optical system for diffused 
illumination (Lieberman, pg. 10, par. 184) 

for homogenously illuminating by diffusing light output from the light source to an 
external illumination plane (Lieberman, Fig. 28), 

and the light source and the spatial light modulation unit are both shared to 
generate a light image subject to projection and generate a light subject to diffused 
illumination (Lieberman, pg. 6, par. 135; light source is a single laser source). 

At the time the invention was made, it would have been obvious to one having 
ordinary skill in the art to modify the mobile communication terminal of Rafii modified by 
Kloba wherein the mobile communication terminal is configured by using a light source, 
a spatial light modulation unit for modulating light output from the light source, and an 
optical system for projection imaging that projects by imaging a light image output from 
the spatial light modulation unit on an external projection screen, the mobile 
communication terminal comprises an optical system for diffused illumination for 
homogenously illuminating by diffusing light output from the light source to an external 
illumination plane and the light source and the spatial light modulation unit are both 
shared to generate a light image subject to projection and generate a light subject to 
diffused illumination, as taught by Lieberman. 
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As one of ordinary skill in the art would appreciate, the suggestion/motivation for 
doing so would have been to provide an expanded input device for a small sized PDA or 
cell phone (Lieberman, pg. 1, par. 9). 

Claim 13 is rejected as being dependent on rejected claim 12 as discussed 
above and further Rafii teaches a mobile communication terminal, the mobile 
communication terminal comprising (Rafii, Fig. 1A, 80): 

a camera unit that generates image data by transforming the light image to 
electric signals (Rafii, col. 2, II. 21-36); 

and an optical system for camera imaging for imaging the light image subject to 
shooting on the camera unit (Rafii, col. 2, II. 21-57); 

wherein the operation detection means is configured by using operation object 
detection means for detecting at least one of position, direction, and movement of an 
operation object operating on the operation-plane image and operation detection data 
generation means for generating operation detection data corresponding to position, 
direction or movement of the operation object based on the detection results of the 
operation detection means (Rafii, col. 2, II. 21-57); 

and the camera unit and the optical system for camera imaging are both shared 
as the operation object detection means (Rafii, col. 2, II. 21-57). 

Claim 16 is rejected as being dependent on rejected claim 15 as discussed 
above and further Rafii modified by Kloba fails to teach a mobile communication 
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terminal wlierein tlie mobile communication terminal is configured by using a light 
source, a spatial light modulation unit for modulating light output from the light source, 
and an optical system for projection imaging that projects by imaging a light image 
output from the spatial light modulation unit on an external projection screen, the mobile 
communication terminal comprises an optical system for diffused illumination for 
homogenously illuminating by diffusing light output from the light source to an external 
illumination plane, and the light source and the spatial light modulation unit are both 
shared to generate a light image subject to projection and generate a light subject to 
diffused illumination. 

Lieberman discloses a virtual data entry device, analogous in art with that of Rafii 
modified by Kloba, wherein the mobile communication terminal is configured by using a 
light source, a spatial light modulation unit for modulating light output from the light 
source, and an optical system for projection imaging that projects by imaging a light 
image output from the spatial light modulation unit (Lieberman, Fig. 28; pg. 10, par. 184) 

on an external projection screen (Lieberman, Fig. 29; pg. 10, par. 186), 

wherein the mobile communication terminal comprises an optical system for 
diffused illumination (Lieberman, pg. 10, par. 184) 

for homogenously illuminating by diffusing light output from the light source to an 
external illumination plane (Lieberman, Fig. 28), 

and wherein the light source and the spatial light modulation unit are both shared 
to generate a light image subject to projection and generate a light subject to diffused 
illumination (Lieberman, pg. 6, par. 135; light source is a single laser source). 



Application/Control Number: 10/586,319 Page 14 

Art Unit: 2629 

At the time the invention was made, it would have been obvious to one having 
ordinary skill in the art to modify the mobile communication terminal of Rafii modified by 
Kloba wherein the mobile communication terminal is configured by using a light source, 
a spatial light modulation unit for modulating light output from the light source, and an 
optical system for projection imaging that projects by imaging a light image output from 
the spatial light modulation unit on an external projection screen, wherein the mobile 
communication terminal comprises an optical system for diffused illumination for 
homogenously illuminating by diffusing light output from the light source to an external 
illumination plane and wherein the light source and the spatial light modulation unit are 
both shared to generate a light image subject to projection and generate a light subject 
to diffused illumination, as taught by Lieberman. 

As one of ordinary skill in the art would appreciate, the suggestion/motivation for 
doing so would have been to provide an expanded input device for a small sized PDA or 
cell phone (Lieberman, pg. 1, par. 9). 

Claim 17 is rejected as being dependent on rejected claim 16 as discussed 
above and further Rafii teaches a mobile communication terminal, the mobile 
communication terminal comprising (Rafii, Fig. 1A, 80): 

a camera unit that generates image data by transforming the light image to 
electric signals (Rafii, col. 2, II. 21-36); 

and an optical system for camera imaging for imaging the light image subject to 
shooting on the camera unit (Rafii, col. 2, II. 21-57); 
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wherein the operation detector is configured by using an operation object 
detector that detects at least one of position, direction, and movement of an operation 
object operating on the operation-plane image and operation detection data generator 
that generates operation detection data corresponding to position, direction or 
movement of the operation object based on the detection results of the operation object 
detector (Rafii, col. 2, II. 21-57); 

and the camera unit and the optical system for camera imaging are both shared 
as the operation object detector (Rafii, col. 2, II. 21-57). 

Response to Arguments 

Applicant's arguments filed 07/31/2009 with respect to claim 13 have been 
considered but are moot in view of the new ground(s) of rejection. 

Applicant argues on page 1 1 of applicant's response that an examination of claim 
1 3 was omitted from the first Office Action. Examiner agrees with applicant and claim 
13 is now included in this second, non-final Office Action. 

Applicant's arguments filed 07/31/2009 with respect to claims 4, 6 and 7 have 
been fully considered but they are not persuasive. 

Applicant argues on page 14 of the applicant's response that the independent 
claims have been amended to include the feature that the terminal projects one of a 
plurality of predefined operation-plane images, and that by having the input layouts pre- 
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progrannnned into tlie device, the application/applet developer does not need to design 
and integrate image layouts into their software, thus reducing the development time and 
complexity. 

Rafii discloses a predefined operation-plane layout (Rafii, Fig. 1A,1B; col. 4, II. 8- 
50), and Kloba discloses downloadable web content to be used by a mobile device in an 
interactive manner (Kloba, Abstract). 

Rafii teaches a user that is enabled to input commands and data from a passive 
virtual emulation of a manual input device, as in the predefined operation-plane images 
projected in Fig. 1 A and 1 B. The mobile device of Rafii uses light to project a 
predefined grid or image onto a work surface in front of the device. Likewise, Kloba 
teaches a system, method, and computer program product for customizing channels, 
content, and data for mobile devices that is downloadable from the internet for use in a 
predefined operation-plane image for the application programs already residing in the 
mobile device. 

Therefore, Rafii, as well as Kloba, teach a predefined operation-plane layout. 
Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHARLES HICKS whose telephone number is 571-270- 
7535. The examiner can normally be reached on Monday-Thursday from 7:00 to 4:30. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Sumati Lefkowitz, can be reached on 571-272-3638. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



/Sumati Lefkowitz/ 

Supervisory Patent Examiner, Art Unit 2629 



